Internet Info 1994 March/Internet Info CD-ROM (Walnut Creek) (March 1994).iso/answers/news/inn-faq/diff3

Internet Message Format | 1994-04-05 | 19KB

Path: bloom-beacon.mit.edu!hookup!swrinde!cs.utexas.edu!uunet!news.mentorg.com!sdl!not-for-mail
From: [e-mail] (Tom Limoncelli)
Newsgroups: news.software.nntp,news.software.b,news.answers
Subject: Changes to INN FAQ Part 3/4: Operational and Misc. Questions
Supersedes: <[e-mail]>
Followup-To: news.software.nntp
Date: 5 Apr 1994 11:37:14 -0400
Organization: Mentor Graphics - IC Group, Warren, NJ, USA
Lines: 508
Sender: [e-mail]
Approved: [e-mail]
Distribution: world
Expires: 04/20/94
Message-ID: <[e-mail]>
References: <[e-mail]>
Reply-To: [e-mail] (Tom Limoncelli)
NNTP-Posting-Host: sdl.warren.mentorg.com
Xref: bloom-beacon.mit.edu news.software.nntp:5495 news.software.b:4177 news.answers:17712

Posted-By: auto-faq 2.4
Archive-name: inn-faq/diff3

No differences encountered
--
Tom Limoncelli -- [e-mail] (work) -- [e-mail] (play)
"Psst!  Hey, Anthony!  Y'know what I        | Disclaimer:  I do not
like about existing?"  "Uh... uh... what?"  | speak for Mentor Graphics.
"Possessing a physical extension."  -TSA    |

*** 1.48    1994/01/17 14:30:24
--- FAQ-inn.3    1994/03/31 21:14:54
***************
*** 1,4 ****
! Last Changed: $Id: FAQ-inn.3,v 1.48 1994/01/17 14:30:24 tal Exp $

                    Part 3 of 4

--- 1,4 ----
! Last Changed: $Id: FAQ-inn.3,v 1.61 1994/03/15 20:09:03 tal Exp $

                    Part 3 of 4

***************
*** 32,37 ****
--- 32,39 ----
      Suddenly my active and history files are owned by root!
      How come my host name comes out twice in the Path line?
      Expire had problems last night, and while I fixed the problem,
+     expire reports "Group not matched (removed?) --
+         Using default expiration"
      Why doesn't this newsfeeds entry do what I want?
      Why am I forwarding cancel messages for articles in comp.foo
      Debugging someone that is feeding you.
***************
*** 42,55 ****
      Directories are being created with wrong permissions.
      Why am I getting alt.sex.pictures even though I have
      More about the "to.*" groups
  HOW DO I... (Big changes you can make to the system):
      How do I set up a delayed IHAVE/SENDME over NNTP?
      Can I use gzip with INN?
      What do I do if /var/spool/news is set up over many partitions?
  BUGS IN 1.4:
!     (this section currently empty)
  BUGS IN 1.3:
      7-bit encoded batches are not correctly processed. Why is this?
  BUGS IN 1.2:
      Why doesn't nntpget work?

--- 44,65 ----
      Directories are being created with wrong permissions.
      Why am I getting alt.sex.pictures even though I have
      More about the "to.*" groups
+     What's a decent syslog.conf configuration?
+     INN batcher writing "#!rnews 0" separators
+     Posting while throttled doesn't work
+     "innd: overview! spooling" and the file "overview!" in /usr/spool/news/out.going
+     "newgroup" control messages aren't being executed
  HOW DO I... (Big changes you can make to the system):
      How do I set up a delayed IHAVE/SENDME over NNTP?
      Can I use gzip with INN?
      What do I do if /var/spool/news is set up over many partitions?
+     archiving expired articles
  BUGS IN 1.4:
!     Security Patch
!     Looping Select Patch
  BUGS IN 1.3:
      7-bit encoded batches are not correctly processed. Why is this?
+     NOV (overchan) doesn't work well.
  BUGS IN 1.2:
      Why doesn't nntpget work?

***************
*** 74,79 ****
--- 84,93 ----

  Subject:  Safe way to edit the "active" file?

+ First of all, you could manipulate the active file using the ctlinnd
+ "newgroup", "rmgroup" and "changegroup" commands.  However, sometimes
+ you just need to do a lot of editing all at once:
+
  The following sequence is the shortest:

      ctlinnd pause "edit active"
***************
*** 81,87 ****
  ctlinnd reload active "edit active"
  ctlinnd go "edit active"

! Simple!  No need to "flush", the "pause" does that.


  ------------------------------
--- 95,142 ----
      ctlinnd reload active "edit active"
      ctlinnd go "edit active"

! Simple!  No need to "flush" since the "pause" does that.
!
! > What if I need to delete 3000 lines from my active file?
!
! I would definately edit the active file manually (using
! the above procedure).
!
! > What if I need to delete 10 lines from my active file?
!
! For a couple quick changes, I recommend using "ctlinnd".  This is a
! little slow because all channels are closed and reopened after each
! "rmgroup", "newgroup", and "changegroup".  However, it's easier than
! remembering the above sequence.
!
! DO NOT THROTTLE THE SERVER WHEN DOING MULTIPLE rmgroup COMMANDS.  There
! is a bug in INN (all versions) that will shred your active file if you
! do multiple "rmgroup" messages while the server if throttled.  This is
! a common mistake.  People think the "rmgroup"'s will go faster if the
! server is throttled.  It will go faster, it will also shred your active
! file.
!
! If you have a large number of groups to remove or create, you can use
! awk to write a script to do the work for you.
!
!     % cat thelist
!     alt.foo.bar
!     alt.delete.me
!     comp.sys.mac
!     % awk <thelist '{ print "ctlinnd rmgroup " $1 }'
!     ctlinnd rmgroup alt.foo.bar
!     ctlinnd rmgroup alt.delete.me
!     ctlinnd rmgroup comp.sys.mac
!
! Now, you can either send the output of that to "| sh -x",
! or you can redirect the output to a file, and "source" the
! file.
!
! If you want to create a bunch of newsgroups, the awk
! command might be like this:
!
!     % awk <thelist \
!     '{ print "ctlinnd newgroup " $1 " y user@host" }' | sh -x


  ------------------------------
***************
*** 132,139 ****

  STEP 6: When you feel you are ready to install the new $INN/site files:

!     % cd $INN/site
!     % make install

  STEP 7: Re-start the system:

--- 187,194 ----

  STEP 6: When you feel you are ready to install the new $INN/site files:

!     # cd $INN/site
!     # make install

  STEP 7: Re-start the system:

***************
*** 140,146 ****
      % sh /usr/lib/news/etc/rc.news

  STEP 8: If everything was done right you should be up and running.
! Part ??? of the FAQ gives tips on testing your configuration.


  ------------------------------
--- 195,201 ----
      % sh /usr/lib/news/etc/rc.news

  STEP 8: If everything was done right you should be up and running.
! Part 2 of the FAQ gives tips on testing your configuration.


  ------------------------------
***************
*** 211,218 ****

  ------------------------------

! Subject:  Expire had problems last night, and while I fixed the problem,
!       it still won't run.

  When expire starts up it "reserves" the server so that nobody else can
  pause or throttle it.  This prevents anyone else from coming in and
--- 266,273 ----

  ------------------------------

! Subject:  Expire had problems last night, and while I fixed the
!             problem, it still won't run.

  When expire starts up it "reserves" the server so that nobody else can
  pause or throttle it.  This prevents anyone else from coming in and
***************
*** 225,230 ****
--- 280,300 ----

  ------------------------------

+ Subject: expire reports "Group not matched (removed?) --
+         Using default expiration"
+
+ Expire says:
+ Group not matched (removed?) alt.techno-shamanism -- Using default expiration
+ Group not matched (removed?) misc.computers.forsale -- Using default expiration
+ Group not matched (removed?) de.rec.sf.startrek -- Using default expiration
+
+ That just means that you've removed those newsgroups groups and expire
+ is slowly removing articles from the spool as they expire.  Eventually
+ the articles will all have been deleted and so will these messages.
+
+
+ ------------------------------
+
  Subject:  Why doesn't this newsfeeds entry do what I want?
              "foo.com:alt,!alt.sex"

***************
*** 369,377 ****
       it.  If a control message is posted to a  group  whose  name
       ends  with  the  four characters ``.ctl'' then the suffix is
       stripped off and what is left is used  as  the  group  name.
!      For  example,  a cancel message posted to ``news.admin.ctl''
       will be sent to all sites that subscribe to  ``control''  or
!      ``news.admin.''

  There is also a pointer to this in newsfeeds(5).

--- 439,447 ----
      it.  If a control message is posted to a  group  whose  name
      ends  with  the  four characters ``.ctl'' then the suffix is
      stripped off and what is left is used  as  the  group  name.
!     For  example,  a cancel message posted to ``news.admin.misc.ctl''
      will be sent to all sites that subscribe to  ``control''  or
!     ``news.admin.misc''.

  There is also a pointer to this in newsfeeds(5).

***************
*** 380,385 ****
--- 450,593 ----
  Yes.


+ ------------------------------
+
+ Subject:  What's a decent syslog.conf configuration?
+
+ The configuration will be different for each site, but here is what
+ Greg Earle recommends as the lines for the "news.*" related part.
+ Remember that most syslog's require tabs, not spaces.
+
+ Greg's canonical SunOS 4.1.x INN-related syslog.conf entries (which can
+ be merged into your current configuration):
+
+ #
+ # INN stuff
+ #
+ ##  Send critical messages to everyone who is logged in and to the console.
+ news.crit               *
+ news.crit               /dev/console
+
+ ##  Log news messages to separate files.
+ ##  Note that each level includes all of the above it.
+ ## =()<news.crit        @<_PATH_MOST_LOGS>@/news.crit>()=
+ news.crit               /var/log/news/news.crit
+ ## =()<news.err         @<_PATH_MOST_LOGS>@/news.err>()=
+ news.err                /var/log/news/news.err
+ ## =()<news.notice      @<_PATH_MOST_LOGS>@/news.notice>()=
+ news.notice             /var/log/news/news.notice
+
+
+ ------------------------------
+
+ Subject:  INN batcher writing "#!rnews 0" separators
+
+ >Outgoing UUCP batches from here are going out with "#!rnews 0" at
+ >the head of each article.
+
+ Most common cause:  your newsfeeds entry has "Wnm" not "Wnb".
+
+ Other reasons:
+
+ batchfiles have something other than a single space between article
+ filename and size
+
+ batchfiles lack size information (all the articles sizes will be read
+ from the batch file as zero)
+
+
+ ------------------------------
+
+ Subject:  Posting while throttled doesn't work
+
+ >I want to be able to allow my users to be able to post articles when
+ >innwatch has throttled the system when the spool disk is "full".
+
+ Cannot be done in 1.4.
+
+ At a minimum, in 1.5 nnrpd will spool the post for the user.
+
+
+ ------------------------------
+
+ Subject:  "innd: overview! spooling" and the file "overview!" in /usr/spool/news/out.going
+
+ > About once a month or so, I get the following warning messages:
+ >
+ > Jan 20 07:20:22 optima innd: overview!:31:proc:9193 cant flush count 14639 Operation would block
+ > Jan 20 07:20:22 optima innd: overview! spooling 14639 bytes
+ >
+ > And there's a file "overview!" in /usr/spool/news/out.going with stuff in it.
+ >
+ > Should I be doing anything more with this than ignoring it, and maybe
+ > occasionally deleting it (it just grows)?
+
+ This happens because innd is feeding info to overchan faster than
+ overchan can process it.  The overflow is sent to the file
+ "overview!".  This file can be deleted, as nnrpd will grab the missing
+ data out of the articles "manually".  The slow-down won't be noticed.
+ However, you can "expireover -a" to "fill in the holes".
+
+ To prevent this in the future, you need to make overchan run faster.
+ This is easy to do.  I found that moving the overview files into
+ /usr/spool/news/over.view made things fast enough that the problem went
+ away.  Change "_PATH_OVERVIEWDIR" in config.data, recompile, and you're
+ done.  (You will need to recompile any newsreaders that read via NFS or
+ off the local disk).
+
+ Why does that speed up overchan?  overchan works by opening the proper
+ .overview file, appending 1 line to it, then closing the file.  If you
+ have the .overview file in the same directory as 10000 articles then
+ opening the .overview file will take a huge amount of time.  The open()
+ call literally searches though an average 5000 (half of 10000) file
+ names to find ".overview".  If you move your .overview files so that
+ each one is in it's own directory, (say,
+ /usr/spool/news/over.view/{group}/{name}/.overview) then open() is
+ searching through 3 files ( ".", "..", and ".overview") to find 1
+ file.  ( O(N/2) where N=10000 vs.  N=3... and you thought those
+ freshman year CS classes would never be useful!)
+
+ There isn't much you can do to make the "append" and "close" steps much
+ faster, except maybe install a PrestoServe or similar write-cache, and
+ that won't help very much.
+
+ Profiling overchan (with PureSoft's Quantify product) found that the
+ open() (actually searching the directory to find the .overview file)
+ was around 80% of the execution time of overchan.  That was reduced to
+ 40% when I moved the .overview files to their own directory.  With the
+ change, overview's profiling statistics are pretty flat. (which is
+ good).
+
+ DO run "expireover -a" to fix the problem.
+
+ DO NOT try feeding the "overview!" file to overchan manually.
+     (1) overchan doesn't do any locking and you'll have two overchan's
+         running at once.
+     (2) overchan only appends to the .overview files.  If you've gotten
+         any articles since the "overview!" file was created (you will
+         have) then you'll be appending told old entries that are out of
+         order.  Your ".overview" files must be in sorted order for the
+         other utilities to work right.
+
+
+ ------------------------------
+
+ Subject:  "newgroup" control messages aren't being executed
+
+ > "newgroup" control messages aren't be executed
+
+ The usual blame for this is _PATH_EGREP points to a grep that doesn't
+ understand regular expressions.  For example, GNU grep only understands
+ regular expressions if it is called "egrep" (i.e. not "gnuegrep" or
+ "egnugrep").
+
+ Make sure you have a link or symlink between egnugrep and egrep.  You
+ then need to modify config.data so that _PATH_EGREP is
+ /your/local/path/egrep and NOT /your/local/path/egnuegrep.  Then
+ recompile and "make install" to have the new binaries and shell
+ scripts installed.
+
+
  =====================================================================
          HOW DO I... (Big changes you can make to the system)
  =====================================================================
***************
*** 555,568 ****
          find . -type l -print | grep '[0-9][0-9]*' | xargs -t rm


  =====================================================================
                               BUGS IN 1.4
  =====================================================================

! (this section currently empty, it will soon have the 2-3 unofficial
! patches that are sometimes needed.)


  =====================================================================
                               BUGS IN 1.3
  =====================================================================
--- 763,827 ----
          find . -type l -print | grep '[0-9][0-9]*' | xargs -t rm


+ ------------------------------
+
+ Subject:  archiving expired articles
+
+ In <2hmomh$[e-mail]> [e-mail] writes:
+ >What options do I have in INN for archiving local newsgroups?
+
+ See doc/archive.8.  You could also put "never:never:never" in your
+ expire.ctl file.
+
+ >Any help would be appreciated. Any cookbook examples would also help.
+
+ A cookbook newsfeeds entry:
+     # Feed all moderated source postings to an archiver
+     source-archive!:!*,*sources*,!*wanted*,!*.d\
+         :Tc,Nm,Wn:/usr/local/bin/archive -f -i /usr/spool/news.archive/INDEX
+
+
+ ------------------------------
+
+ Subject:  How do I set up inpaths with INN?
+
+ inpaths should work just fine with INN as it ships.  However, you can
+ make it run faster by using the following shell script.  Edit it to
+ your tastes.  It replaces the long "(cd /var/spool/news ;
+ /usr/local/bin/gfind . -type f -print | /usr/lib/news/local/inpaths sdl
+ | /usr/ucb/mail admin,[e-mail] )" which people usually
+ use.
+
+ #!/bin/sh
+ . /home/news/innshellvars
+ cd ${SPOOL}
+ awk '(NR > 2){print $3}' < ${HISTORY} | tr . / | sort | \
+ inpaths `innconfval pathhost` | \
+ ${MAILCMD} [e-mail]
+
+
  =====================================================================
                               BUGS IN 1.4
  =====================================================================

! UNOFFICIAL patches for INN 1.4 are available via anonymous FTP at
! ftp://ftp.pop.psu.edu/pub/src/news/inn-patches/
!
! The file ftp://ftp.pop.psu.edu/pub/src/news/inn-patches/README describes
! each of the UNOFFICIAL patches.
!
! The ones that are highly recommended are:

+ 1.4-to-1.4sec     -- Fixes the major security hole in INN.
+ 1.4sec-to-1.4sec2 -- Fixes the remaining known security hole in INN.
+
+ select-loop-bug.patch -- Under some circumstances innd can lose track
+ of a file descriptor and end up sitting in a select() loop.  If your
+ INN suddenly is using up tons of CPU time and not getting much done,
+ install this UNOFFICIAL patch.  Some OSs are more susecptible to this
+ bug.

+
  =====================================================================
                               BUGS IN 1.3
  =====================================================================
***************
*** 580,585 ****
--- 839,853 ----
  fixed in INN 1.4.


+ ------------------------------
+
+ Subject:  NOV (overchan) doesn't work well.
+
+ Correct.  The NOV support in 1.3 didn't have all the bugs worked out.
+ Don't use NOV under INN 1.3.  Better yet, upgrade to 1.4sec and get all
+ the benefits!
+
+
  =====================================================================
                               BUGS IN 1.2
                   (Hey, it's 1994!  Upgrade already!)
***************
*** 589,595 ****

  Subject:  Why doesn't nntpget work?

! The nntpget in INN 1.2 doesn't work.  Period.  Upgrade to the latest version
! of INN.


--- 857,863 ----

  Subject:  Why doesn't nntpget work?

! The nntpget in INN 1.2 doesn't work.  Period.  Upgrade to the latest
! version of INN.